Hiba Khatib, Emily Leibfritz, David Venator, Jazmyn Lu
Published
March 13, 2023
Abstract
This file contains the code for the project on the Titanic, as part of the STAT303-2 course in Winter 2023.
#importing all necessary libraries import pandas as pdimport numpy as npimport seaborn as snsimport matplotlib.pyplot as pltimport statsmodels.api as smimport statsmodels.formula.api as smfimport pandas as pdimport numpy as npimport statsmodels.formula.api as smimport seaborn as snsimport matplotlib.pyplot as pltimport itertoolsimport timefrom statsmodels.stats.outliers_influence import variance_inflation_factorfrom statsmodels.tools.tools import add_constantfrom sklearn.model_selection import train_test_splitfrom sklearn.linear_model import LogisticRegressionfrom sklearn.metrics import ConfusionMatrixDisplay, classification_reportfrom sklearn import linear_modelfrom sklearn.metrics import mean_squared_errorfrom tqdm import tnrange, tqdm_notebookfrom sklearn.metrics import (confusion_matrix, accuracy_score)import matplotlib.pyplot as pltfrom sklearn.datasets import make_classificationfrom sklearn.metrics import confusion_matrix, ConfusionMatrixDisplayfrom sklearn.model_selection import train_test_splitfrom sklearn.svm import SVCfrom sklearn import datasets, metrics, model_selection, svmfrom sklearn.metrics import precision_recall_curve
C:\Users\david\anaconda3\lib\site-packages\statsmodels\tsa\base\tsa_model.py:7: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
from pandas import (to_datetime, Int64Index, DatetimeIndex, Period,
C:\Users\david\anaconda3\lib\site-packages\statsmodels\tsa\base\tsa_model.py:7: FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
from pandas import (to_datetime, Int64Index, DatetimeIndex, Period,
0.1 Data quality check / cleaning / preparation
0.1.1 Data quality check
By Jazmyn Lu
train = pd.read_csv('../train.csv') #taking a quick peek at the data!train.head()
PassengerId
Survived
Pclass
Name
Sex
Age
SibSp
Parch
Ticket
Fare
Cabin
Embarked
0
1
0
3
Braund, Mr. Owen Harris
male
22.0
1
0
A/5 21171
7.2500
NaN
S
1
2
1
1
Cumings, Mrs. John Bradley (Florence Briggs Th...
female
38.0
1
0
PC 17599
71.2833
C85
C
2
3
1
3
Heikkinen, Miss. Laina
female
26.0
0
0
STON/O2. 3101282
7.9250
NaN
S
3
4
1
1
Futrelle, Mrs. Jacques Heath (Lily May Peel)
female
35.0
1
0
113803
53.1000
C123
S
4
5
0
3
Allen, Mr. William Henry
male
35.0
0
0
373450
8.0500
NaN
S
train.isnull().sum() #there seems to be a lot of missing values of age and cabin
PassengerId 0
Survived 0
Pclass 0
Name 0
Sex 0
Age 177
SibSp 0
Parch 0
Ticket 0
Fare 0
Cabin 687
Embarked 2
dtype: int64
train.shape
(891, 12)
There are only 204 observations for “Cabin”, and so it’s probably not worth imputing those data points. Moreover, after some preliminary research, I found that Cabins are just divided by Pclass, a variable we already have, so even if we did include it in the model, they would be correlated variables. What may be informative, however, is imputing values of Age. Let’s explore that:
ax = sns.boxplot(x="SibSp", y ='Age', data=train)ax.figure.set_figwidth(15)
Seems like there is a trend where increased number of siblings and spouses means smaller age
ax = sns.boxplot(x="Parch", y ='Age', data=train)ax.figure.set_figwidth(15)
There also seems to be some correlation
ax = sns.boxplot(x="Pclass", y ='Age', data=train)ax.figure.set_figwidth(15)
Higher class, older people
sns.kdeplot(train['Fare'].values, y=train['Age'].values,color='blue',shade=True )plt.title("Age vs Fare")
Text(0.5, 1.0, 'Age vs Fare')
Some association between Fare and Age
#PClass, Parch, and SibSp variables are categorical, not numerical. Transform to strings so model treats it as categoricaltrain['Pclass'] = train['Pclass'].apply(str)train['Parch'] = train['Parch'].apply(str)train['SibSp'] = train['SibSp'].apply(str)
missing_age = train[train['Age'].isna()] #dataframe containing only rows missing Agehas_age = train.dropna(subset=['Age']) #dataframe containing only no rows missing Ageols_object = smf.ols(formula ='Age~SibSp+Parch+Pclass+Fare', data = has_age) #Making model based on observationsmodel = ols_object.fit()model.summary()
OLS Regression Results
Dep. Variable:
Age
R-squared:
0.305
Model:
OLS
Adj. R-squared:
0.291
Method:
Least Squares
F-statistic:
21.94
Date:
Mon, 13 Mar 2023
Prob (F-statistic):
1.16e-46
Time:
23:01:54
Log-Likelihood:
-2793.2
No. Observations:
714
AIC:
5616.
Df Residuals:
699
BIC:
5685.
Df Model:
14
Covariance Type:
nonrobust
coef
std err
t
P>|t|
[0.025
0.975]
Intercept
43.1668
1.359
31.753
0.000
40.498
45.836
SibSp[T.1]
-1.8618
1.142
-1.630
0.104
-4.104
0.380
SibSp[T.2]
-5.7785
2.561
-2.256
0.024
-10.807
-0.750
SibSp[T.3]
-8.3643
3.748
-2.231
0.026
-15.724
-1.005
SibSp[T.4]
-11.6098
3.178
-3.653
0.000
-17.850
-5.370
SibSp[T.5]
-5.1848
5.774
-0.898
0.369
-16.521
6.151
Parch[T.1]
-5.9267
1.393
-4.253
0.000
-8.663
-3.191
Parch[T.2]
-11.8790
1.808
-6.571
0.000
-15.428
-8.330
Parch[T.3]
5.7309
5.548
1.033
0.302
-5.162
16.624
Parch[T.4]
15.6641
6.217
2.520
0.012
3.458
27.870
Parch[T.5]
12.7629
5.548
2.300
0.022
1.870
23.656
Parch[T.6]
17.5979
12.297
1.431
0.153
-6.545
41.741
Pclass[T.2]
-9.8186
1.502
-6.539
0.000
-12.767
-6.870
Pclass[T.3]
-14.9540
1.431
-10.448
0.000
-17.764
-12.144
Fare
-0.0202
0.011
-1.763
0.078
-0.043
0.002
Omnibus:
37.930
Durbin-Watson:
1.873
Prob(Omnibus):
0.000
Jarque-Bera (JB):
42.648
Skew:
0.571
Prob(JB):
5.48e-10
Kurtosis:
3.359
Cond. No.
1.70e+03
Notes: [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. [2] The condition number is large, 1.7e+03. This might indicate that there are strong multicollinearity or other numerical problems.
Does not violate non-linearity of data, but there does seem to be non-constant variance of error terms. Take sqrt transformation of Age and see if it resolves this issue:
ols_object = smf.ols(formula ='np.sqrt(Age)~SibSp+Parch+Pclass+Fare', data = has_age)model_sqrt_age = ols_object.fit()model_sqrt_age.summary()
OLS Regression Results
Dep. Variable:
np.sqrt(Age)
R-squared:
0.349
Model:
OLS
Adj. R-squared:
0.336
Method:
Least Squares
F-statistic:
26.74
Date:
Mon, 13 Mar 2023
Prob (F-statistic):
3.89e-56
Time:
23:01:55
Log-Likelihood:
-1141.2
No. Observations:
714
AIC:
2312.
Df Residuals:
699
BIC:
2381.
Df Model:
14
Covariance Type:
nonrobust
coef
std err
t
P>|t|
[0.025
0.975]
Intercept
6.5176
0.134
48.479
0.000
6.254
6.782
SibSp[T.1]
-0.1719
0.113
-1.522
0.128
-0.394
0.050
SibSp[T.2]
-0.7000
0.253
-2.764
0.006
-1.197
-0.203
SibSp[T.3]
-1.0391
0.371
-2.803
0.005
-1.767
-0.311
SibSp[T.4]
-1.5136
0.314
-4.816
0.000
-2.131
-0.896
SibSp[T.5]
-0.7256
0.571
-1.271
0.204
-1.847
0.395
Parch[T.1]
-0.9035
0.138
-6.556
0.000
-1.174
-0.633
Parch[T.2]
-1.4262
0.179
-7.977
0.000
-1.777
-1.075
Parch[T.3]
0.4934
0.549
0.899
0.369
-0.584
1.571
Parch[T.4]
1.2739
0.615
2.072
0.039
0.067
2.481
Parch[T.5]
1.1880
0.549
2.165
0.031
0.111
2.265
Parch[T.6]
1.5659
1.216
1.288
0.198
-0.822
3.954
Pclass[T.2]
-0.8557
0.149
-5.762
0.000
-1.147
-0.564
Pclass[T.3]
-1.3142
0.142
-9.284
0.000
-1.592
-1.036
Fare
-0.0009
0.001
-0.751
0.453
-0.003
0.001
Omnibus:
6.406
Durbin-Watson:
1.872
Prob(Omnibus):
0.041
Jarque-Bera (JB):
6.815
Skew:
-0.162
Prob(JB):
0.0331
Kurtosis:
3.352
Cond. No.
1.70e+03
Notes: [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. [2] The condition number is large, 1.7e+03. This might indicate that there are strong multicollinearity or other numerical problems.
R-squared improves a bit. Let’s check the residuals
Better. Not really worth doing any variable transformations since Fare isn’t significant in this model anyways, and the rest of the variables are categorical. As a last step, see if we need to remove outliers, high leverage points, and influential points
Seems to be 4 outliers (studentized residuals larger than 3) and 37 high leverage points. None of them are influential points
Now we can predict age! In the missing_age dataframe, there are 7 rows where ‘SibSp’ = 8, but our model which includes people that have age only includes ‘SibSp’ from 1 to 5. Hence, the model can’t fit those values. Maybe we can hand-impute them?
missing_age.loc[missing_age['SibSp']=='8']
PassengerId
Survived
Pclass
Name
Sex
Age
SibSp
Parch
Ticket
Fare
Cabin
Embarked
159
160
0
3
Sage, Master. Thomas Henry
male
NaN
8
2
CA. 2343
69.55
NaN
S
180
181
0
3
Sage, Miss. Constance Gladys
female
NaN
8
2
CA. 2343
69.55
NaN
S
201
202
0
3
Sage, Mr. Frederick
male
NaN
8
2
CA. 2343
69.55
NaN
S
324
325
0
3
Sage, Mr. George John Jr
male
NaN
8
2
CA. 2343
69.55
NaN
S
792
793
0
3
Sage, Miss. Stella Anna
female
NaN
8
2
CA. 2343
69.55
NaN
S
846
847
0
3
Sage, Mr. Douglas Bullen
male
NaN
8
2
CA. 2343
69.55
NaN
S
863
864
0
3
Sage, Miss. Dorothy Edith "Dolly"
female
NaN
8
2
CA. 2343
69.55
NaN
S
They all seem to come from the same family! Did a little bit of research, and found some information about this family: Stella (born 1891), George John (born 1892), Douglas Bullen (born 1894), Frederick (born 1895), Dorothy Florence (born 1897), Anthony William (born 1899), Elizabeth Ada (born 1901), Constance Gladys (born 1904), Thomas Henry (born 1911). Soure: https://www.nationalarchives.gov.uk/titanic/stories/sage-family.htm
Add these values into “train” dataframe, re-make missing_age and has_age, rerun model, then do prediction!
train.loc[159,'Age']=1train.loc[180,'Age']=8train.loc[201,'Age']=17train.loc[324,'Age']=20train.loc[792,'Age']=21train.loc[846,'Age']=18train.loc[863,'Age']=15missing_age = train[train['Age'].isna()] #dataframe containing only rows missing Agehas_age = train.dropna(subset=['Age']) #dataframe containing only no rows missing Ageols_object = smf.ols(formula ='np.sqrt(Age)~SibSp+Parch+Pclass+Fare', data = has_age)model_age_final = ols_object.fit()model_age_final.summary()
OLS Regression Results
Dep. Variable:
np.sqrt(Age)
R-squared:
0.354
Model:
OLS
Adj. R-squared:
0.341
Method:
Least Squares
F-statistic:
25.80
Date:
Mon, 13 Mar 2023
Prob (F-statistic):
2.59e-57
Time:
23:01:56
Log-Likelihood:
-1152.4
No. Observations:
721
AIC:
2337.
Df Residuals:
705
BIC:
2410.
Df Model:
15
Covariance Type:
nonrobust
coef
std err
t
P>|t|
[0.025
0.975]
Intercept
6.5176
0.135
48.454
0.000
6.253
6.782
SibSp[T.1]
-0.1719
0.113
-1.521
0.129
-0.394
0.050
SibSp[T.2]
-0.7000
0.253
-2.762
0.006
-1.198
-0.202
SibSp[T.3]
-1.0391
0.371
-2.802
0.005
-1.767
-0.311
SibSp[T.4]
-1.5136
0.314
-4.813
0.000
-2.131
-0.896
SibSp[T.5]
-0.7256
0.571
-1.270
0.204
-1.847
0.396
SibSp[T.8]
-0.1291
0.494
-0.261
0.794
-1.099
0.841
Parch[T.1]
-0.9035
0.138
-6.553
0.000
-1.174
-0.633
Parch[T.2]
-1.4262
0.179
-7.973
0.000
-1.777
-1.075
Parch[T.3]
0.4934
0.549
0.899
0.369
-0.584
1.571
Parch[T.4]
1.2739
0.615
2.071
0.039
0.066
2.482
Parch[T.5]
1.1880
0.549
2.164
0.031
0.110
2.266
Parch[T.6]
1.5659
1.217
1.287
0.199
-0.823
3.955
Pclass[T.2]
-0.8557
0.149
-5.759
0.000
-1.147
-0.564
Pclass[T.3]
-1.3142
0.142
-9.280
0.000
-1.592
-1.036
Fare
-0.0009
0.001
-0.750
0.453
-0.003
0.001
Omnibus:
6.916
Durbin-Watson:
1.894
Prob(Omnibus):
0.031
Jarque-Bera (JB):
7.342
Skew:
-0.173
Prob(JB):
0.0255
Kurtosis:
3.353
Cond. No.
1.71e+03
Notes: [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. [2] The condition number is large, 1.71e+03. This might indicate that there are strong multicollinearity or other numerical problems.
missing_age['Age'] = model_age_final.predict(missing_age)**2#predicting based on dataframe with agestrain_with_age = pd.concat([has_age,missing_age]) #putting two dataframes together
C:\Users\david\AppData\Local\Temp/ipykernel_23232/3789173762.py:1: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
missing_age['Age'] = model_age_final.predict(missing_age)**2 #predicting based on dataframe with ages
PassengerId int64
Survived int64
Pclass object
Name object
Sex object
Age float64
SibSp object
Parch object
Ticket object
Fare float64
Embarked object
age_binned category
dtype: object
#Import Statementimport pandas as pdimport numpy as npimport statsmodels.formula.api as smimport seaborn as snsimport matplotlib.pyplot as pltimport itertoolsimport time
#Extract predictors remaining after EDA and preliminary Processingpredictors = train_with_age.columns.tolist()[2:]X = train_with_age[predictors]
#Perform single variable logistic regression, noting which variables throw errorsto_drop = []solo = {}for pred in predictors: factor = predtry: model = sm.logit('Survived~'+ factor, data = train_with_age).fit()#print(model.pvalues) solo[pred] = modelexcept: to_drop.append(pred)print(pred, 'does not model')print(to_drop)
Optimization terminated successfully.
Current function value: 0.608125
Iterations 5
Name does not model
Optimization terminated successfully.
Current function value: 0.515526
Iterations 5
Optimization terminated successfully.
Current function value: 0.663820
Iterations 4
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.641762
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.648598
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Ticket does not model
Optimization terminated successfully.
Current function value: 0.627274
Iterations 6
Optimization terminated successfully.
Current function value: 0.650704
Iterations 4
Optimization terminated successfully.
Current function value: 0.652565
Iterations 5
['Name', 'Ticket']
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
#Check for illogical regressions - pvalues are NaN, or pvalues have an average of 1 (no significance)for key inlist(solo.keys()):if solo[key].pvalues.isna().value_counts()[False] <sum(solo[key].pvalues.isna().value_counts()): to_drop.append(key)elif (solo[key].pvalues >0.99).mean() ==1: to_drop.append(key)
#This shows the columns that either did not regress, or had irregular p-valuesprint(to_drop)
['Name', 'Ticket']
#Drops these columns from analysisX = X.drop(columns = to_drop)for dropped in to_drop:if dropped inlist(solo.keys()):del solo[dropped]
#Plots the single variable regression for all potential predictors after first filter - prints all coeffictients(0) and pvalues(1) for further examiniationplt.figure(figsize=(20,10))plt.rcParams.update({'font.size': 18, 'lines.markersize': 10})for i inrange(len(list(solo.keys()))):print(solo[list(solo.keys())[i]]) plt.subplot(2, 4, i+1)iflist(solo.keys())[i] =='age_binned': sns.scatterplot(train_with_age[list(solo.keys())[i]].apply(lambda x: str(x)), train_with_age['Survived'], color ='orange') #Visualizing data logit_model = solo[list(solo.keys())[i]]print(list(solo.keys())[i], 'params are\n', pd.concat([solo[list(solo.keys())[i]].params, solo[list(solo.keys())[i]].pvalues], axis =1)) sns.lineplot(train_with_age[list(solo.keys())[i]].apply(lambda x: str(x)), logit_model.predict(train_with_age), color ='blue') #Visualizing modelelse: sns.scatterplot(x =list(solo.keys())[i], y ='Survived', data = train_with_age, color ='orange') #Visualizing data logit_model = solo[list(solo.keys())[i]]print(list(solo.keys())[i], 'params are\n', pd.concat([solo[list(solo.keys())[i]].params, solo[list(solo.keys())[i]].pvalues], axis =1)) sns.lineplot(x =list(solo.keys())[i], y = logit_model.predict(train_with_age), data = train_with_age, color ='blue') #Visualizing modelplt.xticks(fontsize=5)plt.savefig('preliminary.png')
<statsmodels.discrete.discrete_model.BinaryResultsWrapper object at 0x0000016DF0A7B7C0>
Pclass params are
0 1
Intercept 0.515813 2.614685e-04
Pclass[T.2] -0.624616 2.240629e-03
Pclass[T.3] -1.655584 5.728842e-21
<statsmodels.discrete.discrete_model.BinaryResultsWrapper object at 0x0000016DF0639D90>
Sex params are
0 1
Intercept 1.047969 4.834782e-16
Sex[T.male] -2.505089 1.076430e-50
<statsmodels.discrete.discrete_model.BinaryResultsWrapper object at 0x0000016DEFD28220>
Age params are
0 1
Intercept -0.238328 0.150413
Age -0.008266 0.112114
<statsmodels.discrete.discrete_model.BinaryResultsWrapper object at 0x0000016DF0AE2C10>
SibSp params are
0 1
Intercept -0.648914 3.339052e-14
SibSp[T.1] 0.792702 1.149214e-06
SibSp[T.2] 0.505813 1.928958e-01
SibSp[T.3] -0.449698 4.410116e-01
SibSp[T.4] -0.960524 1.323208e-01
SibSp[T.5] -21.425465 9.993849e-01
SibSp[T.8] -18.420556 9.971887e-01
<statsmodels.discrete.discrete_model.BinaryResultsWrapper object at 0x0000016DF0AFC430>
Parch params are
0 1
Intercept -0.655657 6.207840e-16
Parch[T.1] 0.859752 2.091348e-05
Parch[T.2] 0.655657 5.842185e-03
Parch[T.3] 1.061122 2.469289e-01
Parch[T.4] -37.658822 9.999997e-01
Parch[T.5] -0.730638 5.145366e-01
Parch[T.6] -9.769780 9.575623e-01
<statsmodels.discrete.discrete_model.BinaryResultsWrapper object at 0x0000016DF0B0B340>
Fare params are
0 1
Intercept -0.939518 4.446465e-23
Fare 0.014973 1.598443e-11
<statsmodels.discrete.discrete_model.BinaryResultsWrapper object at 0x0000016DF0B0B880>
Embarked params are
0 1
Intercept 0.215111 1.657297e-01
Embarked[T.Q] -0.664062 1.792398e-02
Embarked[T.S] -0.891998 4.121688e-07
<statsmodels.discrete.discrete_model.BinaryResultsWrapper object at 0x0000016DF0B6D6A0>
C:\Users\david\anaconda3\lib\site-packages\seaborn\_decorators.py:36: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
warnings.warn(
C:\Users\david\anaconda3\lib\site-packages\seaborn\_decorators.py:36: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
warnings.warn(
model_breakers = []def processSubset(predictor_subset): model = sm.logit('Survived~'+'+'.join(predictor_subset),data = train_with_age).fit() pval = model.prsquaredreturn {"model":model, "llr_pvalue":pval, "vars":predictor_subset}def getBest_model(k): tic = time.time() results = []for combo in itertools.combinations(X.columns, k):#print(list(combo))try: results.append(processSubset((list(combo))))except: model_breakers.append(list(combo))# Wrap everything up in a dataframe models = pd.DataFrame(results)#print(models)# Choose the model with the highest RSS best_model = models.loc[models['llr_pvalue'].argmax()] toc = time.time()print("Processed", models.shape[0], "models on", k, "predictors in", (toc-tic), "seconds.")return k, models#Function to select the best model amongst the best models for 'k' predictors, where k = 1,2,3,..models_best = pd.DataFrame(columns=['num_elements', "model"])tic = time.time()for i inrange(1,1+X.shape[1]):try: models_best.loc[i] = getBest_model(i)except:print('error')toc = time.time()print("Total elapsed time:", (toc-tic), "seconds.")
Optimization terminated successfully.
Current function value: 0.608125
Iterations 5
Optimization terminated successfully.
Current function value: 0.515526
Iterations 5
Optimization terminated successfully.
Current function value: 0.663820
Iterations 4
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.641762
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.648598
Iterations: 35
Optimization terminated successfully.
Current function value: 0.627274
Iterations 6
Optimization terminated successfully.
Current function value: 0.650704
Iterations 4
Optimization terminated successfully.
Current function value: 0.652565
Iterations 5
Processed 8 models on 1 predictors in 0.2000133991241455 seconds.
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Optimization terminated successfully.
Current function value: 0.464850
Iterations 6
Optimization terminated successfully.
Current function value: 0.583569
Iterations 5
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.595705
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.593237
Iterations: 35
Optimization terminated successfully.
Current function value: 0.602351
Iterations 6
Optimization terminated successfully.
Current function value: 0.597483
Iterations 5
Optimization terminated successfully.
Current function value: 0.580545
Iterations 6
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Optimization terminated successfully.
Current function value: 0.515523
Iterations 5
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.496705
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.505102
Iterations: 35
Optimization terminated successfully.
Current function value: 0.496925
Iterations 6
Optimization terminated successfully.
Current function value: 0.504689
Iterations 6
Optimization terminated successfully.
Current function value: 0.506273
Iterations 6
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.637999
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.648595
Iterations: 35
Optimization terminated successfully.
Current function value: 0.623073
Iterations 6
Optimization terminated successfully.
Current function value: 0.648869
Iterations 5
Optimization terminated successfully.
Current function value: 0.652548
Iterations 5
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.618792
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.607042
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.630508
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.623020
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.615182
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.634899
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.642163
Iterations: 35
Optimization terminated successfully.
Current function value: 0.620682
Iterations 6
Optimization terminated successfully.
Current function value: 0.616715
Iterations 6
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Optimization terminated successfully.
Current function value: 0.636455
Iterations 5
Processed 27 models on 2 predictors in 0.944068193435669 seconds.
Optimization terminated successfully.
Current function value: 0.452614
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.454352
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.457629
Iterations: 35
Optimization terminated successfully.
Current function value: 0.464417
Iterations 6
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Optimization terminated successfully.
Current function value: 0.460372
Iterations 6
Optimization terminated successfully.
Current function value: 0.449721
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.567164
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Optimization terminated successfully.
Current function value: 0.581369
Iterations 5
Optimization terminated successfully.
Current function value: 0.573283
Iterations 6
Optimization terminated successfully.
Current function value: 0.580143
Iterations 6
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.577713
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.589845
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.586367
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.590132
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.581904
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.576619
Iterations: 35
Optimization terminated successfully.
Current function value: 0.592728
Iterations 6
Optimization terminated successfully.
Current function value: 0.578695
Iterations 6
Optimization terminated successfully.
Current function value: 0.570089
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.495613
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
Current function value: 0.504866
Iterations: 35
Optimization terminated successfully.
Current function value: 0.496602
Iterations 6
Optimization terminated successfully.
Current function value: 0.504669
Iterations 6
Optimization terminated successfully.
Current function value: 0.506251
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.482451
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.475594
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.490649
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.480061
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.484649
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.495705
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.495889
Iterations: 35
Optimization terminated successfully.
Current function value: 0.491886
Iterations 6
Optimization terminated successfully.
Current function value: 0.490085
Iterations 6
Optimization terminated successfully.
Current function value: 0.494974
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.618126
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.596205
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.626676
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.622871
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.613264
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.634892
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.642108
Iterations: 35
Optimization terminated successfully.
Current function value: 0.616474
Iterations 6
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Optimization terminated successfully.
Current function value: 0.616582
Iterations 6
Optimization terminated successfully.
Current function value: 0.636338
Iterations 5
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.589286
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.608982
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.610130
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.602332
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.583717
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.610764
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.608994
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.608540
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.627273
Iterations: 35
Optimization terminated successfully.
Current function value: 0.608772
Iterations 6
Processed 50 models on 3 predictors in 2.2131640911102295 seconds.
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.435569
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.446585
Iterations: 35
Optimization terminated successfully.
Current function value: 0.452602
Iterations 6
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Optimization terminated successfully.
Current function value: 0.448690
Iterations 6
Optimization terminated successfully.
Current function value: 0.449309
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.444769
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.453059
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.451475
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.429273
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.457005
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.453986
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.442906
Iterations: 35
Optimization terminated successfully.
Current function value: 0.460188
Iterations 6
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Optimization terminated successfully.
Current function value: 0.449720
Iterations 6
Optimization terminated successfully.
Current function value: 0.445568
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.558741
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.563716
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.558977
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.560686
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.575596
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.574789
Iterations: 35
Optimization terminated successfully.
Current function value: 0.571653
Iterations 5
Optimization terminated successfully.
Current function value: 0.578295
Iterations 6
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Optimization terminated successfully.
Current function value: 0.569484
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.573778
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.568164
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.555325
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.581805
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.557601
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.552589
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.579890
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.564788
Iterations: 35
Optimization terminated successfully.
Current function value: 0.568754
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.482372
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.471413
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.487331
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.479860
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.484283
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.495649
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.495888
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Optimization terminated successfully.
Current function value: 0.491525
Iterations 6
Optimization terminated successfully.
Current function value: 0.489966
Iterations 6
Optimization terminated successfully.
Current function value: 0.494895
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.462568
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.476362
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.469797
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.457823
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.471835
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.481035
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.475954
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.485769
Iterations: 35
Optimization terminated successfully.
Current function value: 0.484153
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.608280
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.610130
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.592177
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.583219
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.610465
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.607277
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.608494
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.627273
Iterations: 35
Optimization terminated successfully.
Current function value: 0.608527
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.584658
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.575430
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.598948
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.578429
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.601547
Iterations: 35
Processed 64 models on 4 predictors in 3.312241554260254 seconds.
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.431108
Iterations: 35
Optimization terminated successfully.
Current function value: 0.435045
Iterations 28
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.433822
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.428519
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.446342
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.443568
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.442475
Iterations: 35
Optimization terminated successfully.
Current function value: 0.448682
Iterations 6
Optimization terminated successfully.
Current function value: 0.449309
Iterations 6
Optimization terminated successfully.
Current function value: 0.445062
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.443593
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.442731
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.424918
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.450651
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.428715
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.427162
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.453700
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.442670
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.439526
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Optimization terminated successfully.
Current function value: 0.445512
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.556020
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.550081
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.554919
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.556278
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.556837
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.551689
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.565911
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.564443
Iterations: 35
Optimization terminated successfully.
Current function value: 0.568156
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.565379
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.552311
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.546354
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.549470
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.460388
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.476165
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.469750
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.468905
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.457361
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.471545
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.480637
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.475784
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.485741
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Optimization terminated successfully.
Current function value: 0.483987
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.462855
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.455169
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.471431
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.575184
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.577813
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.601443
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.570301
Iterations: 35
Processed 48 models on 5 predictors in 3.0112199783325195 seconds.
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.430436
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.429709
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.424328
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.433509
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.427939
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.426355
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.443497
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.442210
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.439026
Iterations: 35
Optimization terminated successfully.
Current function value: 0.445011
Iterations 6
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.441950
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.423878
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.423098
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.426827
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.439456
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.548143
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.551825
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.545810
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.548530
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.544043
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.458844
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.446409
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1810: RuntimeWarning: overflow encountered in exp
return 1/(1+np.exp(-X))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py:1863: RuntimeWarning: divide by zero encountered in log
return np.sum(np.log(self.cdf(q*np.dot(X,params))))
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: inf
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.454671
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.471225
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.444918
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.569984
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Processed 26 models on 6 predictors in 1.7341265678405762 seconds.
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.429270
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.423219
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.422460
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.426002
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.438941
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.422368
Iterations: 35
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.543423
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.444487
Iterations: 35
Processed 8 models on 7 predictors in 0.5540387630462646 seconds.
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.421673
Iterations: 35
Processed 1 models on 8 predictors in 0.07800459861755371 seconds.
Total elapsed time: 12.06287956237793 seconds.
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
#Plots all generated models based on r-squared - clear separation is seendef best_sub_plots(): plt.figure(figsize=(20,10)) plt.rcParams.update({'font.size': 18, 'lines.markersize': 10})# Set up a 2x2 grid so we can look at 4 plots at once#plt.subplot(2, 2, 1)# We will now plot a red dot to indicate the model with the largest adjusted R^2 statistic.# The argmax() function can be used to identify the location of the maximum point of a vector plot_array = []for i inrange(1, models_best.shape[0]+1):#print(i) models_made = models_best.loc[models_best['num_elements'] == i]['model'][i]for k inrange(models_made.shape[0]): plot_array.append([i, models_made['model'], models_made['llr_pvalue'][k], models_made['vars'][k]]) data = pd.DataFrame(plot_array, columns = ['num_elements', 'model', 'rsquared', 'vars']) plt.plot(data['num_elements'], data['rsquared'], 'ro') plt.xlabel('# Predictors') plt.ylabel('rsquared') plt.ylim(bottom=0) plt.savefig('rsquared_all_models.png')return dataall_models = best_sub_plots()
#Creates a datefram including the model, its psuedo r-squared, the variables considered, and booleans for whether a given variable was useddef new_col(var, dfr):return dfr['vars'].apply(lambda x: var in x)for variable inlist(X.columns): all_models[variable] = new_col(variable, all_models)
#Shows the percentage of models including a given veriable when the pseudo r-squared is over 0.2 (boundary between regions)all_models[all_models.rsquared >0.2][list(X.columns)].sum()/all_models['Sex'].sum()
Pclass 0.525000
Sex 1.000000
Age 0.508333
SibSp 0.466667
Parch 0.508333
Fare 0.491667
Embarked 0.500000
age_binned 0.516667
dtype: float64
#Checks correlation of all numeric valuesfrom statsmodels.stats.outliers_influence import variance_inflation_factorfrom statsmodels.tools.tools import add_constantX_vif = add_constant(train_with_age[X_v])vif_data = pd.DataFrame()vif_data["feature"] = X_vif.columnsfor i inrange(len(X_vif.columns)): vif_data.loc[i,'VIF'] = variance_inflation_factor(X_vif.values, i)print(vif_data)
feature VIF
0 const 5.950504
1 Age 1.011726
2 Fare 1.011726
C:\Users\david\anaconda3\lib\site-packages\statsmodels\tsa\tsatools.py:142: FutureWarning: In a future version of pandas all arguments of concat except for the argument 'objs' will be keyword-only.
x = pd.concat(x[::order], 1)
0.4 Developing the model
All of the folllowing code produced by Hiba Khatib & Emily Leibfritz
Try a basic model with only the variables that were deemed as relevant in the previous variable selection code.
final_train = train_with_age.copy()
# dropping the columns that we don't needfinal_train = train_with_age.drop(columns = ['PassengerId', 'Name', 'Ticket'])
final_train.head()
Survived
Pclass
Sex
Age
SibSp
Parch
Fare
Embarked
age_binned
0
0
3
male
22.0
1
0
7.2500
S
(20.315, 30.263]
1
1
1
female
38.0
1
0
71.2833
C
(30.263, 40.21]
2
1
3
female
26.0
0
0
7.9250
S
(20.315, 30.263]
3
1
1
female
35.0
1
0
53.1000
S
(30.263, 40.21]
4
0
3
male
35.0
0
0
8.0500
S
(30.263, 40.21]
# don't thinl we need to include this# # basic statsmodel# logit_model1 = sm.logit(formula = 'Survived~ Pclass + Sex + Age + SibSp + Parch + Fare + Embarked + age_binned', data = final_train).fit()# logit_model1.summary()
# to work with sklearn, convert Sex, Embarked into dummie variablesfinal_train = pd.get_dummies(final_train, columns = ['Sex'], drop_first =True)final_train = final_train.rename(columns={"Sex_1": "Sex"})final_train.rename(columns={"Sex_male": "Sex"}, inplace =True)final_train['Embarked'] = final_train['Embarked'].apply(lambda x: 1if x =='S'else2if x =='C'else0)# final_train['Pclass'] = pd.to_numeric(final_train['Pclass'])# final_train['SibSp'] = pd.to_numeric(final_train['SibSp'])# final_train['Parch'] = pd.to_numeric(final_train['Parch'])## done later in previous docfinal_train = final_train.drop('age_binned', axis =1)
# split the data to test and train for sklearn modely = final_train.SurvivedX = final_train.drop('Survived', axis =1)X_train, X_test, y_train, y_test = train_test_split(X, y, test_size =0.15, random_state =45)
y_pred_test = sklearn_model.predict(X_test)print('Accuracy of logistic regression on test set : {:.4f}'.format(sklearn_model.score(X_test, y_test)))
Accuracy of logistic regression on test set : 0.7910
The first model with the basic, selected variables has an accuracy of 79.1%
0.4.1 Polish that model by using extensive feature selection on all columns besides the ones discared in previous variable selection
def fit_logistic_reg(X,Y):#Fit linear regression model and return RSS and R squared values model_k = linear_model.LogisticRegression() model_k.fit(X,Y) RSS = mean_squared_error(Y,model_k.predict(X)) *len(Y) R_squared = model_k.score(X,Y)return RSS, R_squared
#Importing tqdm for the progress barfrom tqdm import tnrange, tqdm_notebookk =11RSS_list, R_squared_list, feature_list = [],[], []numb_features = []#Looping over k = 1 to k = 11 features in Xfor k in tnrange(1,len(X_train.columns) +1, desc ='Loop...'):#Looping over all possible combinations: from 11 choose kfor combo in itertools.combinations(X_train.columns,k): tmp_result = fit_logistic_reg(X_train[list(combo)],y_train) #Store temp result RSS_list.append(tmp_result[0]) #Append lists R_squared_list.append(tmp_result[1]) feature_list.append(combo) numb_features.append(len(combo)) #Store in DataFramedf = pd.DataFrame({'numb_features': numb_features,'RSS': RSS_list, 'R_squared':R_squared_list,'features':feature_list})
C:\Users\david\AppData\Local\Temp/ipykernel_23232/3621587686.py:9: TqdmDeprecationWarning: Please use `tqdm.notebook.trange` instead of `tqdm.tnrange`
for k in tnrange(1,len(X_train.columns) + 1, desc = 'Loop...'):
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
When the number of features is 6, the RSS is minimized and the R squared is maximized. This improves the fit of the model. The features that improve the model the best are (Pclass, Age, SibSp, Parch, Embarked, Sex). This means we are dropping the Fare cost.
# Build a model using those variablesX_train = X_train.drop('Fare', axis =1)X_test = X_test.drop('Fare', axis =1)sklearn_model = LogisticRegression()sklearn_model.fit(X_train, y_train)y_pred_test = sklearn_model.predict(X_test)print('Accuracy of logistic regression on test set : {:.4f}'.format(sklearn_model.score(X_test, y_test)))
Accuracy of logistic regression on test set : 0.7985
# make model using thesesklearn_model_1 = LogisticRegression()sklearn_model_1.fit(X_train1, y_train)
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
LogisticRegression()
y_pred_test = sklearn_model_1.predict(X_test1)print('Accuracy of logistic regression on test set : {:.4f}'.format(sklearn_model_1.score(X_test1, y_test)))
Accuracy of logistic regression on test set : 0.8060
0.4.3 Improve more by using forward variable selection including 2 fold interactions between columns
# reload entire data# split the data to test and train for sklearn model again bc X_train now includes all interactionsfinal_train = train_with_age.copy()final_train = train_with_age.drop(columns = ['PassengerId', 'Name', 'Ticket'])# to work with sklearn, convert Sex, Embarked into dummie variablesfinal_train = pd.get_dummies(final_train, columns = ['Sex'], drop_first =True)final_train = final_train.rename(columns={"Sex_1": "Sex"})final_train.rename(columns={"Sex_male": "Sex"}, inplace =True)final_train['Embarked'] = final_train['Embarked'].apply(lambda x: 1if x =='S'else2if x =='C'else0)final_train['Pclass'] = pd.to_numeric(final_train['Pclass'])final_train['SibSp'] = pd.to_numeric(final_train['SibSp'])final_train['Parch'] = pd.to_numeric(final_train['Parch'])final_train = final_train.drop('age_binned', axis =1)y = final_train.SurvivedX = final_train.drop('Survived', axis =1)X_train, X_test, y_train, y_test = train_test_split(X, y, test_size =0.15, random_state =45)
#Creating a dataframe with all the predictorsX = X_train#Since 'X' will change when we include interactions, we need a backup containing all individual predictorsX_backup = X_train# make test dataframefinal_test = pd.concat([X_test.reset_index(drop=True),y_test.reset_index(drop=True)], axis=1)
#Including 2-factor interactions of predictors in train and 'X'. Note that we need train to develop the model, and X to #find 'k' variable subsets from amongst all the predictors under considerationfor combo in itertools.combinations(X_backup.columns, 2): final_train['_'.join(combo)] = final_train[combo[0]]*final_train[combo[1]] final_test['_'.join(combo)] = final_test[combo[0]]*final_test[combo[1]] X.loc[:,'_'.join(combo)] = final_train.loc[:,'_'.join(combo)]
#Initialization variablesk =11remaining_features =list(X.columns.values)features = []RSS_list, R_squared_list = [np.inf], [np.inf] #Due to 1 indexing of the loop...features_list =dict()for i inrange(1,k+1): best_RSS = np.inffor combo in itertools.combinations(remaining_features,1): RSS = fit_logistic_reg(X[list(combo) + features],y_train) #Store temp result if RSS[0] < best_RSS: best_RSS = RSS[0] best_R_squared = RSS[1] best_feature = combo[0]#Updating variables for next loop features.append(best_feature) remaining_features.remove(best_feature)#Saving values for plotting RSS_list.append(best_RSS) R_squared_list.append(best_R_squared) features_list[i] = features.copy()
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
# create X_train for best by BICX_train_int = X[['Age_Sex','Pclass_SibSp','Pclass_Embarked','Parch','Age_Parch','SibSp','Pclass_Parch','Fare_Sex']]# make this for X_test as wellX_test_int = final_test[['Age_Sex','Pclass_SibSp','Pclass_Embarked','Parch','Age_Parch','SibSp','Pclass_Parch','Fare_Sex']]sklearn_model_int = LogisticRegression()sklearn_model_int.fit(X_train_int, y_train)
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\linear_model\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
LogisticRegression()
y_pred_test_int = sklearn_model_int.predict(X_test_int)print('Accuracy of logistic regression on test set : {:.4f}'.format(sklearn_model_int.score(X_test_int, y_test)))
Accuracy of logistic regression on test set : 0.8209
# should we remove this?import matplotlib.pyplot as pltfrom sklearn.datasets import make_classificationfrom sklearn.metrics import confusion_matrix, ConfusionMatrixDisplayfrom sklearn.model_selection import train_test_splitfrom sklearn.svm import SVCdisp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=sklearn_model_1.classes_)disp.plot()plt.show()
from sklearn import datasets, metrics, model_selection, svmmetrics.plot_roc_curve(sklearn_model_int, X_test_int, y_test)plt.show()
C:\Users\david\AppData\Roaming\Python\Python39\site-packages\sklearn\utils\deprecation.py:87: FutureWarning: Function plot_roc_curve is deprecated; Function :func:`plot_roc_curve` is deprecated in 1.0 and will be removed in 1.2. Use one of the class methods: :meth:`sklearn.metric.RocCurveDisplay.from_predictions` or :meth:`sklearn.metric.RocCurveDisplay.from_estimator`.
warnings.warn(msg, category=FutureWarning)
ypred_int = sklearn_model_int.predict(X_train_int)from sklearn.metrics import precision_recall_curvey = y_train# ypred = logit_model_diabetes.predict(train)p, r, thresholds = precision_recall_curve(y, ypred_int)def plot_precision_recall_vs_threshold(precisions, recalls, thresholds): plt.figure(figsize=(8, 8)) plt.title("Precision and Recall Scores as a function of the decision threshold") plt.plot(thresholds, precisions[:-1], "b--", label="Precision") plt.plot(thresholds, recalls[:-1], "g-", label="Recall") plt.ylabel("Score") plt.xlabel("Decision Threshold") plt.legend(loc='best') plt.legend()plot_precision_recall_vs_threshold(p, r, thresholds)
Warning: Maximum number of iterations has been exceeded.
Current function value: 0.428660
Iterations: 35
C:\Users\david\anaconda3\lib\site-packages\statsmodels\base\model.py:566: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
warnings.warn("Maximum Likelihood optimization failed to "
Logit Regression Results
Dep. Variable:
Survived
No. Observations:
889
Model:
Logit
Df Residuals:
864
Method:
MLE
Df Model:
24
Date:
Mon, 13 Mar 2023
Pseudo R-squ.:
0.3556
Time:
23:02:38
Log-Likelihood:
-381.08
converged:
False
LL-Null:
-591.41
Covariance Type:
nonrobust
LLR p-value:
4.255e-74
coef
std err
z
P>|z|
[0.025
0.975]
Intercept
3.7933
0.649
5.842
0.000
2.521
5.066
Pclass[T.2]
-0.7546
0.946
-0.798
0.425
-2.608
1.099
Pclass[T.3]
-1.9508
0.765
-2.550
0.011
-3.450
-0.452
SibSp[T.1]
0.0824
0.226
0.365
0.715
-0.361
0.525
SibSp[T.2]
-0.2729
0.552
-0.494
0.621
-1.355
0.809
SibSp[T.3]
-2.1850
0.708
-3.086
0.002
-3.573
-0.797
SibSp[T.4]
-1.5553
0.798
-1.949
0.051
-3.119
0.009
SibSp[T.5]
-16.6591
2409.518
-0.007
0.994
-4739.228
4705.910
SibSp[T.8]
-151.1821
2.44e+32
-6.2e-31
1.000
-4.78e+32
4.78e+32
Parch[T.1]
0.3193
0.293
1.090
0.276
-0.255
0.893
Parch[T.2]
0.0553
0.382
0.145
0.885
-0.694
0.805
Parch[T.3]
0.4009
1.074
0.373
0.709
-1.703
2.505
Parch[T.4]
-24.1570
1.13e+05
-0.000
1.000
-2.22e+05
2.21e+05
Parch[T.5]
-1.1479
1.186
-0.968
0.333
-3.473
1.177
Parch[T.6]
-10.0291
164.873
-0.061
0.951
-333.174
313.115
Embarked[T.Q]
-1.0361
2.008
-0.516
0.606
-4.971
2.899
Embarked[T.S]
-0.1784
0.357
-0.500
0.617
-0.878
0.521
Sex[T.male]
-2.6932
0.206
-13.051
0.000
-3.098
-2.289
Embarked[T.Q]:Pclass[T.2]
2.0320
2.828
0.719
0.472
-3.510
7.574
Embarked[T.S]:Pclass[T.2]
0.0862
0.755
0.114
0.909
-1.394
1.567
Embarked[T.Q]:Pclass[T.3]
0.9013
2.057
0.438
0.661
-3.130
4.933
Embarked[T.S]:Pclass[T.3]
-0.3563
0.514
-0.693
0.488
-1.364
0.651
Age
-0.0342
0.014
-2.498
0.012
-0.061
-0.007
Age:Pclass[T.2]
-0.0180
0.021
-0.876
0.381
-0.058
0.022
Age:Pclass[T.3]
-0.0024
0.020
-0.118
0.906
-0.042
0.037
0.7 Conclusions and Recommendations to stakeholder(s)
1 You may or may not have code to put in this section. Delete this section if it is irrelevant.
As we are analyzing a historical event, we do not have any recommendations to present as there are no stakeholders that would significantly benefit from our insights.
In terms of conclusions, out optimized logistic model summary shows the factors that were the most impactful. The first thing that stands out, as it did throughout our analysis, was the impact of sex on survival. The coefficient for male was -2.7381, with a p-value rounding to ~0. This shows the that the stereotypical prioritization of women in survival situtations is historically accurate, with female survival rate being significantly higher. The second thing, that once again was in line with our expectations, was the dependance on class. The intercept coefficent, corresponding to class 1 passengers, has coefficent of 4.0693 with a p-value ~0 showing that the upper class passengers where able to survive at a significantly higher rate, especially compared to 3rd class (ceof = -1.9166, pval = 0.009). Interestingly the second class coefficient was not singificant, highlighting the gap between the highest and lowest class.
Another aspect that we expected to be significant was Age, to emphasize the expression ‘women and children first’ that was frequently stated in the movie, and is stereotypical of emergency situations even today. We were able to find a significant decrease in survival with Age (pval = 0.008), but the coefficient was just -0.0349, a lower impact than expected. Bucketing age did not appear useful, although in early analysis showed that this trend (lower survival with age) is not consistent across age groups. Another factor that may give insight into this area is the factors SubSp, and Parch, which show the number of siblings aboard and number of parents and children. The former was shown to have a moderate impact of -0.3319, with a pvalue 0.004. This indicates that larger families (children on board with more siblings) had more difficulty surviving, potentially skewing the age to have less impact than expected.
Other factors lead to improved modeling outcomes, but did not carry significant in the model, making their impact less intuitive to understand. In general our analysis found that the stereotypical prioritization of women and children appears to exist, but gender seems to be a much larger predictor than age. Class was shown to be second only to gender, showing how even in crisis, those with higher class are able to secure better outcomes. These factors give insight into human nature, and confirm many aspects of people are traditional thought to react in emergencies.
Overall, our model stil has shortcomings, with a recall of just 0.68, showing the difficulty in correctly identifying survived passengers when a clear majority did not survie, and in an emergency situation things are inherently unpredictable, to a degree. If this model were to be improved further, the focus would be on the precisions and recall for the survival values, as those are the minority.